Approximation Bounds for Smooth Functions in C(IRd) by Neural and Mixture Networks

We consider the approximation of smooth multivariate functions in C(I R d) by feedforward neural networks with a single hidden layer of non-linear ridge functions. Under certain assumptions on the smoothness of the functions being approximated and on the activation functions in the neural network, we present upper bounds on the degree of approximation achieved over the domain IR d , thereby generalizing available results for compact domains. We extend the approximation results to the so-called mixture of expert architecture, which has received wide attention in recent years, showing that the same type of approximation bound may be achieved.